refactor(collections)!: consolidate platform backlog collections into project-planning - #2602
refactor(collections)!: consolidate platform backlog collections into project-planning#2602Bill Berry (WilliamBerryiii) wants to merge 14 commits into
Conversation
… project-planning Azure DevOps, GitHub, Jira, and GitLab each carried a collection, a backlog manager agent, and near-duplicate copies of the same backlog protocol. What actually differs between trackers is the command surface, field vocabulary, reference-ID prefix, and action verbs; everything structural was common. Backlog capability now lives in project-planning, with the shared protocol owned by skills and each platform expressed as a reference delta. - retire the ado, github, jira, and gitlab collections - add backlog-management, backlog-plan, backlog-execute, functional-planner - add backlog-manager and functional-planner agents - retire 7 agents, 19 prompts, and 18 instruction files - move ADO pull request and build workflows into skill references - replace ado-backlog and github-backlog guides with unified backlog docs - rewire 12 collection manifests and regenerate plugin and reference output BREAKING CHANGE: the ado, github, jira, and gitlab collections are removed. Install project-planning instead. The four Marketplace identities are orphaned; existing installs are not redirected. Fixes #2601 💥 - Generated by Copilot
- delete four platform package entries and their package docs - register backlog manager, functional planner, and six skills in project-planning - relocate gh-code-scanning to security with its experimental maturity - drop 52 dangling component paths from hve-core-all and project-planning - preserve owasp-docker removed tombstones in security and hve-core-all BREAKING CHANGE: the ado, github, jira, and gitlab packages no longer exist. Backlog capabilities consolidate into project-planning, which resolves the backing tracker at runtime. 📦 - Generated by Copilot
- strip ado write, github write, and terminal tools from Backlog Manager - add ado, github, and jira backlog executor subagents with one surface each - declare executors in agents frontmatter and retarget mutating handoffs - add direct-invocation contract to backlog-execute with write-surface stop - register the three executors in project-planning and hve-core-all BREAKING CHANGE: Backlog Manager no longer mutates any tracker. Every create, update, link, transition, close, and comment is dispatched to the executor for the resolved platform. 🔐 - Generated by Copilot
…ndings
* Add always-on backlog-guardrails instruction over all three tracking roots
* Make backlog-management the only owner of autonomy tiers and sanitization guards
* Gate ADO System.State as a transition and pause on GitHub security labels
* Replace cross-package #file: imports and hard-coded Jira CLI paths with named resolution
* Add Success criteria and Stop rules to the three user-invocable skills
BREAKING CHANGE: `backlog-guardrails.instructions.md` now attaches to
`.copilot-tracking/{workitems,github-issues,jira-issues}/**` and stops any
tracker-bound mutation when `backlog-management` does not resolve. Azure DevOps
`System.State` changes gate under Partial autonomy where they previously
auto-executed as field updates.
🔐 Generated by Copilot
* Wrap Get-ScopeDir call site in @() so zero and single scopes stay arrays * Revert the bypassable Windows drive-letter exemption to the base disallow * Add stimuli for functional-planner and the three platform executors * Narrow a conformance grader that matched a deleted instruction filename * Rename the synthetic inventory fixture to project-planning/backlog-manager BREAKING CHANGE: surface signatures regenerate with the strict Windows disallow, so `evals/baseline-equivalence/surface-signatures/backlog-manager.yml` changes on the next generator run. 🔧 Generated by Copilot
* Retype backlog-plan and backlog-execute as skills across lifecycle guides * List only genuinely auto-activated instructions in the delivery stage * Route feature prioritization to requirements-author, not backlog-manager * Add capability-status column, text alternatives, and expanded abbreviations * Refresh ms.date on 18 authored pages and reset the PR body counts BREAKING CHANGE: documentation now describes the packages model. The collections vocabulary, the four retired package pages, and the `docs/getting-started/collections.md` route are gone with no redirect. 📝 Generated by Copilot
* Restructure the three executors to the canonical subagent pattern * Delegate the shared mutating protocol to backlog-execute, removing a second copy * Lead Backlog Manager and Functional Planner with success criteria and stop rules * Add Relationship Semantics and Interaction Templates to github and jira references * Retarget two eval stimuli and one test fixture off deleted instructions BREAKING CHANGE: executors now activate `backlog-execute` and run its Required Flow. An executor dispatched without that skill resolvable stops instead of carrying its own copy of the mutating protocol. 🔍 Generated by Copilot
…ogy-consolidation
7248865 to
16d520b
Compare
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.OpenSSF ScorecardScorecard details
Scanned Files
|
…ogy-consolidation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2602 +/- ##
==========================================
+ Coverage 82.85% 86.98% +4.13%
==========================================
Files 166 104 -62
Lines 22508 11792 -10716
Branches 29 29
==========================================
- Hits 18648 10257 -8391
+ Misses 3857 1532 -2325
Partials 3 3
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Eval Execution✅ Status: Passed — no merge-blocking failures (18 advisory assertion failure(s) present)
|
* Cover the instruction P03 introduced, which had no eval stimulus CI requires every changed AI artifact to declare a stimulus. The four Eval Execute jobs share a coverage preflight, so this one gap failed all of them. 🧪 Generated by Copilot
* Supply each constraint in the prompt instead of assuming the agent is loaded * Grade applied behavior rather than recall of an unloaded artifact * Tag every agent stimulus advisory per the vally-tests agent routing rule * Drop three graders that scored narration rather than measurable behavior The agent-behavior spec loads two fixed skills and never loads the .agent.md under test; --tag only filters stimuli. Graders needing artifact recall scored 0/5 while prompt-answerable and negative graders scored 5/5. 🧪 Generated by Copilot
…dation' into feat/collection-topology-consolidation
Emmeline Hoops (chaosdinosaur)
left a comment
There was a problem hiding this comment.
Comprehensive review found seven blocking High-severity issues. Each blocker is attached inline; Medium and Low findings are grouped in follow-up category comments.
| "agents/hve-core/subagents/vally-test-author.md" | ||
| ], | ||
| "commands": [ | ||
| "commands/hve-core/ado-create-pull-request.md", |
There was a problem hiding this comment.
[High] Standalone hve-core omits a required skill
Problem: This package exposes both ADO commands, but its skills list does not include backlog-management. Both commands require that skill and stop before any Azure DevOps call when it cannot resolve, so the standalone hve-core installation advertises commands that deterministically fail.
Suggested fix: Add skills/project-planning/backlog-management and its recipe closure to hve-core, or move these commands to a package that already supplies the dependency. Add a package-closure test that resolves every command's named skill dependencies in each package that exposes it.
|
|
||
| ### Protocol | ||
|
|
||
| 1. **Audit.** Run `printenv | grep -i JIRA` and classify each variable as set or missing. Use no modifying command during the audit. Check for an existing `~/.jira.env`. |
There was a problem hiding this comment.
[High] Jira setup audit can expose live credentials
Problem: printenv | grep -i JIRA prints complete values for JIRA_API_TOKEN or JIRA_PAT before presentation-layer masking can apply. Terminal output is visible to the agent and may be retained in chat or debug logs, exposing a reusable tracker credential.
Suggested fix: Probe an explicit allowlist of variable names and emit only SET or MISSING. Never print token values or prefixes. Add a sentinel-token test asserting that credentials never appear in stdout, stderr, chat output, or audit logs.
| 2. Resolve and display the **absolute** path so the user knows exactly which file to edit. | ||
| 3. Open it with `code ~/.jira.env`. | ||
| 4. The user replaces the placeholders and saves. | ||
| 5. Source it (`set -a && source ~/.jira.env && set +a`) before running any command. |
There was a problem hiding this comment.
[High] Jira dotenv loading executes configuration as shell code
Problem: Sourcing ~/.jira.env executes substitutions, redirections, and arbitrary commands from a user-writable file in a terminal that holds Jira credentials and repository access. A modified or malformed file can therefore execute code or exfiltrate secrets during setup.
Suggested fix: Parse an allowlisted KEY=VALUE format without shell evaluation, reject unknown or duplicate keys and shell metacharacters, pass values directly to the Jira subprocess environment, and require owner-only permissions. Test $(), backticks, semicolons, redirections, and embedded newlines.
| --- | ||
| name: Functional Planner | ||
| description: 'Read-only Product Manager agent that analyzes PRDs and plans Azure DevOps, GitHub, or Jira work-item hierarchies without mutating a tracker' | ||
| tools: ['execute/getTerminalOutput', 'execute/runInTerminal', 'read/problems', 'read/readFile', 'read/terminalSelection', 'read/terminalLastCommand', 'edit/createDirectory', 'edit/createFile', 'edit/editFiles', 'search', 'web', 'agent', 'ado/search_workitem', 'ado/wit_get_work_item', 'ado/wit_get_work_items_for_iteration', 'ado/wit_list_backlog_work_items', 'ado/wit_list_backlogs', 'ado/wit_list_work_item_comments', 'ado/work_list_team_iterations', 'github/get_me', 'github/list_issue_types', 'github/get_label', 'github/search_issues', 'github/issue_read', 'microsoft-docs/*'] |
There was a problem hiding this comment.
[High] The read-only Functional Planner retains arbitrary shell authority
Problem: The planner is described as strictly read-only but receives execute/runInTerminal. It consumes untrusted PRDs, tracker bodies, comments, and web content, so prompt injection can induce shell commands, credential reads, or Jira mutations. The enforceable capability boundary is broader than the documented role.
Suggested fix: Remove terminal execution and terminal-output access. Route Jira reads through a constrained structured interface exposing only search, get, comments, and fields. Add an adversarial test proving that shell and mutation capabilities are unavailable, not merely refused in prose.
| name: Jira Backlog Executor | ||
| description: "Runs the Jira skill CLI in one confirmed project. Applies a dispatched Jira operation set and returns Jira reads the caller cannot perform." | ||
| tools: | ||
| - execute/runInTerminal |
There was a problem hiding this comment.
[High] The Jira executor's CLI-only policy is not enforced
Problem: execute/runInTerminal grants a general shell while the executor relies on prose to limit it to scripts/jira.py. The executor processes untrusted dispatches, Jira bodies, comments, and CLI output while credentials are present, so injected content can request arbitrary commands or cross-system actions.
Suggested fix: Replace shell access with a dedicated Jira tool wrapper using structured arguments and direct process invocation. Enforce a command allowlist, argument separation, destination checks, bounded output, and credential redaction. If that boundary cannot be provided, do not expose this executor as write-capable.
|
|
||
| Pre-requisite setup: activate the `backlog-execute` skill by name. It owns the shared mutating protocol, including the operation contract, dry-run behavior, resumable execution, and the upstream human-review gate. When it does not resolve, report that the execution protocol is unavailable and stop before any Azure DevOps call. | ||
|
|
||
| 1. Verify the contract: confirm the destination is present and every operation names a supported Azure DevOps action verb. Stop and report if either fails. |
There was a problem hiding this comment.
[High] ADO mutations are not bound to the confirmed project
Problem: Contract validation checks that a destination exists and the verb is supported, but it does not prove that each numeric System.Id or relationship endpoint belongs to the confirmed project. A stale or attacker-influenced handoff can mutate an accessible item in another project while reporting the confirmed destination.
Suggested fix: Hydrate every existing target and relationship endpoint before mutation and verify System.TeamProject matches the confirmed project. Require the confirmed project on creates, reject mismatches and dependent operations, and record the verified binding in the operation log.
| Pre-requisite setup: activate the `jira` skill by name to resolve its CLI entry point, then activate the `backlog-execute` skill, which owns the shared mutating protocol including the operation contract, dry-run behavior, resumable execution, and the upstream human-review gate. When either does not resolve, report which one and stop before any terminal execution. | ||
|
|
||
| 1. Preflight credentials: confirm `JIRA_BASE_URL` and either `JIRA_API_TOKEN` or `JIRA_PAT` are set. Report the missing variable by name and stop; never prompt for a token value in conversation and never echo a credential. | ||
| 2. Verify the contract: confirm the project key is present and every operation maps to a documented CLI command. |
There was a problem hiding this comment.
[High] Jira mutations are not bound to the confirmed project key
Problem: Validation checks only that a project key is present and a CLI command exists. Existing issue keys and create payloads are not required to match the confirmed project, so a handoff confirmed for one project can update, transition, comment on, or create content in another project when the credential has access.
Suggested fix: Normalize the confirmed key, require every existing issue key and hydrated issue project to match it, and require every create payload's fields.project.key to match. Reject the operation set before the first mutation on any mismatch, with tests for create, update, transition, and comment.
Medium/Low findings: security, authority, and data governanceGitHub executor has uncontracted coding-agent delegationProblem: Suggested fix: Remove the tool. If assignment is required, create a separate explicitly confirmed workflow with validated repository/base ref, sanitized instructions, an autonomy gate under every tier, and delegation audit logging. Post-call checkpointing can replay successful mutationsProblem: The workflow performs a remote mutation before checking the handoff operation and appending its result. A failure after remote success but before local persistence can cause resume to repeat creates, comments, links, transitions, closes, or non-idempotent updates. Suggested fix: Write a pending operation record with a stable idempotency key before the call, then persist the remote identifier and terminal state. Reconcile pending operations against the tracker before retrying and add crash-injection tests. Execution logs omit authority and approval provenanceProblem: Logs record a reference, action, and returned key, but not actor identity, confirmed destination, source handoff revision, autonomy tier, approval evidence, sanitization outcomes, validated target binding, correlation ID, or precondition state. Suggested fix: Define a structured append-only audit record with those fields and test audit completeness and resume consistency. Task-planning retention lacks privacy lifecycle controlsProblem: Local artifacts may preserve comments, exact errors, stack traces, names, email addresses, customer identifiers, incident details, or secrets without PII classification, redaction, purpose limitation, access protection, retention, deletion, or operator notice. Suggested fix: Classify and redact PII and probable secrets before persistence, retain only planning-relevant fields, define local access and retention/deletion behavior, and test representative sensitive payloads. |
Medium/Low findings: functional contracts and validationGitHub Actions build-info routing has no reachable toolsProblem: The Backlog Manager advertises direct GitHub Actions run, job, and log queries, but declares only GitHub issue and pull-request tools and prohibits terminal workarounds. The documented branch cannot execute. Suggested fix: Declare concrete read-only Actions tools and test the route, or mark GitHub Actions build info unsupported until that tool surface exists. Windows leakage validation only rejects the C driveProblem: The signature generator claims drive-letter coverage but hard-codes Suggested fix: Use a case-insensitive The shared core skill lacks explicit success and stop contractsProblem: Suggested fix: Add those sections, including stops for unresolved platform/destination, failed preflight, unavailable required skill, human-review triggers, and unresolved temporary IDs. Evals do not exercise enforceable tool boundariesProblem: Refreshed against head Suggested fix: Add contained tests with instrumented fake tools and adversarial PRD, issue, comment, CLI-output, and handoff content. Assert exact allowed calls, zero forbidden calls, destination binding, approval gating, guard application, dry-run behavior, idempotence, and audit records. |
Medium/Low findings: accessibility and labelsHTML checklist state is exposed only as punctuationProblem: Azure DevOps Server HTML uses ordinary list items prefixed with Suggested fix: Use explicit The PRD handoff label contains a replacement characterProblem: Suggested fix: Use |
Medium/Low findings: migration and PR readinessRetired public surfaces lack a complete migration mapProblem: The overview does not map retired packages, agents, prompts, Jira/GitLab skills, and code scanning to installable replacements or state the required reinstall/upgrade action. Suggested fix: Add one authoritative old-to-new table and link it from the PR description and retirement pages. Relocated ADO commands are described as unchangedProblem: Their names remain, but package ownership, dependency loading, preflight, destination confirmation, sanitization, autonomy handling, and failure behavior changed. Suggested fix: Describe name compatibility separately from the new runtime controls and stop behavior on both ADO migration pages. The current branch is behind and required checks failProblem: Head Suggested fix: Update from Required validation checklist entries remain uncheckedProblem: The PR body still leaves local validation, docs validation, spell checking, and link validation unchecked despite broad documentation and generated-surface changes. Suggested fix: Record exact successful command outcomes and have the author check each item only after it passes. PR description omits or misstates material scopeProblem: It describes GitLab as having a backlog-manager agent, omits multiple agent retirements, executors, code-scanning and ADO prompt moves, settings, and licensing/security changes, and leaves linting configuration unchecked despite Suggested fix: Correct the opening and scope list, and align the Type of Change selections with the actual diff. |
Medium/Low findings: documentation contract accuracyDiscovery guide hard-codes ADO artifact namesProblem: It tells GitHub and Jira users to expect Suggested fix: Document platform-resolved filenames or use generic placeholders with an explicit binding table. Execution guide publishes the wrong persistent artifactsProblem: It promises Suggested fix: Document the implemented in-place contract and identify which artifact owns approval, operation history, and completion status. ADO item-type discovery is overstatedProblem: The guide says types are discovered from the process template, but the canonical reference says no MCP tool lists them and user confirmation or an unvalidated marker is required. Suggested fix: State that GitHub/Jira types are discoverable and ADO process types require explicit confirmation. Sprint guide lists files the protocol never createsProblem: It promises Suggested fix: Document actual outputs, or add the missing outputs to the owning contract before documenting them. Task-planning guide uses nonexistent paths and filenamesProblem: It uses Suggested fix: Replace the tree with the implemented bindings and Stage 2 handoff name. Triage guide promises files the workflow does not produceProblem: It lists Suggested fix: Document |
Jamie Kim (jkim323)
left a comment
There was a problem hiding this comment.
Thanks for the exceptionally thorough PR description and consolidation work! The relocation map, change-shape breakdown, explicit capability ownership, and validation evidence made a very large cross-cutting refactor much easier to review. I left couple items to be looked at.
| ### Dry Run Mode | ||
|
|
||
| Dry run is a full simulation with zero platform mutations. When the caller enables `dryRun`: | ||
|
|
||
| * Resolve, validate, and sanitize every payload exactly as a live run would, including the Content Sanitization Guards. | ||
| * Do not call any create, update, transition, close, comment, or link operation. Read-only calls used for validation remain permitted. | ||
| * Assign a simulated key of the form `{{TEMP-N}} -> (dry-run)` instead of a real item key, and mark every dependent operation that would have consumed a real key. | ||
| * Log each operation in `handoff-logs.md` with status `dry-run` and the payload summary that would have been sent. | ||
| * Leave `handoff.md` checkboxes unchecked, because no operation completed. | ||
| * Report the simulated counts and state clearly that nothing was created, changed, or closed. | ||
|
|
||
| Autonomy gates still apply in dry run so the simulated run exercises the same decision path as the live run. | ||
|
|
||
| ### Step 1: Initialize or Resume | ||
|
|
||
| When `handoff-logs.md` exists, read it and `handoff.md`, identify unchecked `[ ]` operations, rebuild the `{{TEMP-N}}` mapping from completed Create entries, and resume from the first unchecked operation. When it does not exist, create it from the template, populate the operation-log skeleton from `handoff.md`, and record inputs in the execution summary. | ||
|
|
||
| Validate before processing: confirm the project or repository is set for creates; confirm each referenced existing item can be read with `get` (skip `{{TEMP-N}}` placeholders during reference validation); call `fields` when create payloads use unvalidated item types or field names; apply the Content Sanitization Guards to all platform-bound fields; abort on critical failures such as missing project scope for creates or an authentication failure, and warn and continue on non-critical failures such as an unknown label or milestone. | ||
|
|
There was a problem hiding this comment.
[P1] Keep dry-run state out of resumable execution records
Dry runs write simulated Create results and {{TEMP-N}} -> (dry-run) mappings into handoff-logs.md, but resume rebuilds temporary-ID mappings from completed Create entries in that same log. A subsequent live run can therefore treat simulated work as complete or substitute (dry-run) into a child, link, or other dependent operation; the current sanitization rules do not reject that value.
Suggested fix: write dry-run entries to a separate Dry Run Log section, never to the Operation Log or the Temporary ID Mapping section; rebuild the mapping only from Operation Log entries with status Success; add a stop rule for a mapping that resolves any placeholder to a simulated key.
|
|
||
| ### Step 1: Initialize or Resume | ||
|
|
||
| When `handoff-logs.md` exists, read it and `handoff.md`, identify unchecked `[ ]` operations, rebuild the `{{TEMP-N}}` mapping from completed Create entries, and resume from the first unchecked operation. When it does not exist, create it from the template, populate the operation-log skeleton from `handoff.md`, and record inputs in the execution summary. |
There was a problem hiding this comment.
[P1] Use one durable resume predicate
The protocol currently resumes from the first unchecked operation here, from the first unlogged operation in backlog-execute, and from the first unchecked or unlogged operation in the core skill. Because Step 2 checks handoff.md before appending its log entry, an interruption in that window leaves the operation checked but unlogged: one path skips it while another reruns it, including a completed Create.
Suggested fix: Please make successful live entries in handoff-logs.md the sole resume authority, append that entry before updating the checkbox, and reconcile all three descriptions to the same rule.
|
|
||
| 1. Summarize recommendations in `triage-plan.md` using a table with columns: Item, Summary, Suggested Fields, Suggested Transition, Duplicates, Action. | ||
| 2. Present the plan for review, highlighting high-confidence updates, potential duplicates, ambiguous transitions, and missing project or item-type context. | ||
| 3. Finalize `triage-plan.md` as the reviewable execution contract and record its path in `planning-log.md`. Name `backlog-execute run <triage-plan.md>` as the separate pass that applies any recommendation. Do not execute a recommendation here, and do not issue a mutating platform call from this workflow. |
There was a problem hiding this comment.
[P1] Materialize triage recommendations into an execution handoff
This path names backlog-execute run <triage-plan.md>, but triage-plan.md has no operation checkboxes, stable <PREFIX>NNN references, verb-ordered sections, or temporary-ID allocations. The execution protocol therefore cannot initialize, resume, resolve dependencies, or mark a triage run complete consistently.
Suggested fix: Have Triage Phase 2 materialize its recommendations into a standard handoff.md with the required operation sections, reference IDs, checkboxes, and temporary IDs, then name backlog-execute run <handoff.md> as the execution path after review. triage-plan.md should remain analysis, not an execution contract.
| * **Open Questions**: [Summary] | ||
| ```` | ||
|
|
||
| ### handoff.md |
There was a problem hiding this comment.
[P1] Add an explicit upstream-review gate to generated handoffs
backlog-execute halts only on an unchecked human-review checkbox, but this shared template creates no such checkbox—only unchecked operation entries. As a result, locally generated handoffs bypass the upstream-review requirement; treating the operation boxes as review gates would instead halt every new run.
Suggested fix: Please add a dedicated ## Human Review or ## User Signoff section (for example, - [ ] Reviewed and validated by a qualified human reviewer) and require that specific checkbox to be checked before execution, while leaving operation checkboxes as execution state only.
| | State changes | Carried by an Update to `System.State`, gated as a transition rather than an ordinary field update | | ||
| | Authoring templates | The interaction templates in the Azure DevOps reference of the `backlog-management` skill | | ||
|
|
||
| ## Constraints |
There was a problem hiding this comment.
[P2] Centralize executor-common constraints to prevent drift
The three executor agents repeat several shared execution constraints and the same handoff-logs.md path-formatting rule, despite backlog-execute being described as the shared mutating protocol.
Suggested fix: Please move the truly platform-neutral rules—including executor-authored-text sanitization, untrusted-content handling, shortcut prohibition, operation logging, and workspace-path formatting—into backlog-execute, then have each executor reference that protocol while retaining only its platform-specific deltas.
| | Task Planning | `backlog-plan` skill, `my-work` then `task-plan` mode | | ||
| | Execution | The executor subagent for the resolved platform, dispatched operation set | | ||
| | Single Item | The executor subagent for the resolved platform, single-item dispatch | | ||
| | PRD Planning | Routes to the `functional-planner` skill (read-only hierarchy planning); on completion, the user invokes Execution | |
There was a problem hiding this comment.
[P2] Dispatch PRD Planning to the same target named everywhere else
The handoff definition and PRD-planning classification identify Functional Planner as the target, but this dispatch row routes to the functional-planner skill instead. Agents and skills have different execution surfaces and contracts, so this leaves the actual PRD-planning path ambiguous.
Suggested fix: Please make this row dispatch to the Functional Planner agent, or change the other references if the skill is intended to be the sole entry point.
| | Failed create | The create call returns an error | Log the error, leave the `{{TEMP-N}}` unresolved, skip every dependent operation that references it, continue | | ||
| | Failed update | The update call returns an error | Log the error and the attempted payload, continue | | ||
| | Item not found | A referenced key returns a not-found response | Log the missing key, skip the operation, continue | | ||
| | Rate limited | The platform reports a rate limit | Pause for the platform's reset window, retry with exponential backoff, log the pause, then continue | |
There was a problem hiding this comment.
Question: Is the unbounded rate-limit retry intentional? This row requires exponential-backoff retries but, unlike the transient-network-failure row below, does not define a maximum attempt count, total wait budget, or terminal outcome when the platform remains rate-limited. If retries are intentionally indefinite, could we document that explicitly and explain how the user regains control? Otherwise, should this cap retries/wait time and abort or block the run while preserving handoff-logs.md for resumption?
| | RPI Orchestration | 1 | High | [RPI Documentation](../rpi/README.md) | | ||
| | [Code Review](#code-review) | 3 | Medium | [Code Review](code-review/README.md) | | ||
| | [Backlog Management](#backlog-management) | 2 active | Very High | [Backlog Management](backlog/README.md) | | ||
| | [Project Planning](#project-planning) | 9 | Medium-High | [Project Planning](project-planning/README.md) | |
There was a problem hiding this comment.
[P2] Reconcile the Project Planning agent count with its landing page
This catalog says Project Planning contains 9 agents, but the linked Project Planning page says “Four agents” and lists only four in its Agent Overview. The surrounding catalog prose also implies a broader capability set than that page documents.
Suggested fix: Please align the count and the group page’s inventory so users can tell which agents are included in this category.
| | Item vocabulary | Platform reference, Platform Bindings table | "work item" | "issue" | "issue" | | ||
| | Action verbs | Platform reference, Platform Bindings table | Create, Update, Link, Comment, No Change | Create, Update, Link, Close, Comment, No Change | Create, Update, Transition, Comment, No Change | | ||
|
|
||
| `planning-log.md`, `handoff.md`, and `handoff-logs.md` are constant across platforms. The templates below use `<analysis-file>` and `<plan-file>` where a binding applies; substitute the platform's value when creating the file. |
There was a problem hiding this comment.
[P2] Define the source for in the shared binding table
The templates require substituting , but this binding table has no reference-ID-prefix row. The values only appear later in each platform reference (WI, IS, and JI), so a consumer following this file’s stated resolution path cannot derive the template IDs.
Suggested fix: Please add a Reference-ID prefix binding here, or explicitly direct resolution to the Platform Bindings table in the active platform reference.
Pull Request
Description
Azure DevOps, GitHub, Jira, and GitLab each carried their own package, their own backlog manager agent, and near-duplicate copies of the same backlog protocol — discovery, triage, sprint planning, PRD-to-work-item planning, and execution.
What actually differs between trackers is small: the command surface, the field vocabulary, the reference-ID prefix, and the action verbs. Everything structural was common and duplicated four ways.
This PR consolidates backlog capability into
project-planning. The shared protocol is owned by skills, and each platform contributes only its delta as a skill reference. A singlebacklog-manageragent resolves the target platform before dispatching, so a team using more than one tracker no longer has to pick a collection per tracker.Related Issue(s)
Fixes #2601
Type of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
hve-builderand addressed all actionable findings.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md).github/hooks/*/*.json)evals/)Other:
.ps1,.sh,.py)Changes
Read this first: the diff is mostly relocation
The file count is large because four packages were dissolved and their contents relocated. Most changed paths are the same bytes at a new address. Ninety-eight files carry real edits.
GitHub collapses each rename pair into one entry, so the PR view shows roughly 250 entries rather than 369 paths.
Where things moved
Paths below are relative to
.github/unless they start withdocs/orevals/.SAMEcounts pairs whose content is byte-identical across the move.The two large blocks — 101 pairs across
gitlabandjira— are Python skill packages moving intact: scripts, tests, fuzz corpora, and lockfiles. Only three files in those blocks differ, and each difference is a path string inside aSKILL.mdor corpusREADME.md.What replaced the retired packages
Platform deltas that survive live as skill references under
backlog-management/references/{ado,github,jira}.md, loaded on demand rather than through always-onapplyToglobs.Packages
Ten packages remain in
.github/plugin/marketplace.json.ado,github,jira, andgitlabare retired and their capability folded intoproject-planning.ado,github,jira,gitlabcatalog entries and theirdocs/plugins/pagesmarketplace.jsonpackage recipes,docs/plugins/project-planning.md,.vscode/settings.jsonThe
.vscode/settings.jsonchange is broader than the four retired packages: it reconciles every artifact discovery location with the current directory topology, including accessibility, data-science, and privacy locations. No entry is retained for a directory that does not exist.New skills and agents
backlog-managementbacklog-planbacklog-executefunctional-plannerbacklog-manageragentfunctional-planneragent{ado,github,jira}-backlog-executorsubagentsbacklog-guardrailsinstructionPer-platform deltas live in
backlog-management/references/{ado,github,jira}.md.Retired artifacts
Net totals: agents 64 to 62 (7 deleted, 5 added), prompts 67 to 49, instruction files 79 to 62 (18 deleted, 1 added).
All 18 deleted instruction files carried an
applyToglob, and 14 of those attached to a backlog tracking root (.copilot-tracking/workitems/**,github-issues/**, orjira-issues/**); the remaining four attached elsewhere. Their content is now reached through skill references, so it loads on demand rather than against the always-on context budget. A newbacklog-guardrails.instructions.mdkeeps mutation safety attached to those three roots for consumers that never load a backlog skill. Separately,community-interaction.instructions.mdis relocated frominstructions/github/toinstructions/project-planning/and remains an instruction.Preserved ADO workflows
Pull request creation and build information have no cross-platform equivalent and are preserved as skill references rather than retired:
instructions/hve-core/ado-create-pull-request.instructions.mdbacklog-management/references/ado-pull-request.mdinstructions/hve-core/ado-get-build-info.instructions.mdbacklog-management/references/ado-build-info.mdDocumentation
docs/agents/ado-backlog/anddocs/agents/github-backlog/are replaced by a unifieddocs/agents/backlog/. Azure DevOps pages with no cross-platform equivalent — build monitoring, PR creation, PRD planning — are retained under anAzure DevOps Delivery Workflowssidebar category and linked from the agents catalog.docs/plugins/package pages and Docusaurus configuration are updated.Other
.github/workflows/gh-code-scanning.ymlis corrected to.github/skills/security/gh-code-scanning/, following that skill's relocation.scripts/evals/New-AgentSurfaceSignatures.ps1emits an alternation covering every declared tracking root, which the multi-platformbacklog-managerrequires. Its call site is wrapped in@()so zero-scope and single-scope results stay arrays, and the Windows drive-letter disallow is unchanged frommainbecause the relaxed form could be defeated by unrelated same-line text.Testing
The validation suite ran once against the final tree.
npm run validate:locallint:md-linksnpm run validate:docsnpm run spell-checknpm run test:ps -- -TestPath scripts/tests/evals/Build-AgentBehaviorSpec.ps1 -WhatIfnpx vally lint --eval-spec evals/regression-without-baselinewarnings onlynpm run docs:generate:checkms.datedrift onlylint:md-linksdid not pass locally. Two external URLs returnedStatus: 0, a connection failure rather than a 404: alearn.microsoft.comthreat-modeling page and an OWASP SSRF page. Both live in files this PR does not modify, and every internal link resolved. CI re-checks this lane from a runner with unrestricted egress.Acceptance criteria were independently re-verified rather than accepted from the implementation record: package count, absence of residual retired-package paths across
.github/andextension/,user-invocablevalues on the five affected skills, story-quality levels, sprint-planning platform neutrality, and Docusaurus package coverage.Known limitations
Jira Backlog Executorbecause Jira's only command surface is a CLI reached through a terminal. Azure DevOps and GitHub reads do not need that round trip. This is the accepted cost of keeping the orchestrator free of terminal access.docs/plugins/sidebar positions are non-contiguous after the four package pages were deleted. Docusaurus tolerates gaps.Checklist
Required Checks
AI Artifact Contributions
hve-builderreview mode to review contributionhve-builderreviewRequired Local Checks
npm run validate:localnpm run validate:docsnpm run spell-checknpm run lint:md-linksSecurity Considerations
Content sanitization guards run before any platform-bound mutation, and an inferred platform requires explicit user confirmation before the first mutating call. The
Backlog Managerorchestrator holds no tracker write tool and no terminal tool, so a mutation it is asked to perform directly is unreachable rather than merely disallowed. Each executor carries exactly one platform's write surface.Release Communication
The breaking change is communicated through Conventional Commits, not a hand-edited CHANGELOG. Each commit that removes or narrows a public surface carries a
!marker and aBREAKING CHANGE:footer, so release-please derives the version bump and the release notes from the commit history.Additional Notes
Opened as a draft. This branch is rebased onto current
mainas a single linear history;collections/andplugins/no longer exist there after #2577.The reviewable surface is 98 files. See the change-shape chart at the top of Changes: 119 relocated paths, 69 generated
docs/reference/pages, and 83 retirements need no line-by-line reading. The 37 new and 61 edited files carry every substantive decision.collections/andplugins/no longer appear in this diff — both were removed frommainby #2577.Parallel PR. This is one of two cut from the same working tree, alongside #2599 (artifact portability). Both branch from
mainand do not stack. Whichever merges second should resolve generated conflicts by discarding both sides and re-runningplugin:generate,docs:generate, andextension:preparerather than merging generated files by hand.Known follow-ups, recorded and not blocking:
docusaurus.config.jshardcodes the package list and omitsinstallerandrpi— pre-existing at merge-basedocs:generatestampsms.dateon every page it touches regardless of content change, requiring a manual revert pass after every regenerationdocs/agents/project-planning/arch-diagram-builder.mdandsecurity-plan-creator.mddocument agents that no longer exist — pre-existing at merge-base, not introduced here